What Circuit Classes Can Be Learned with Non-Trivial Savings?

Despite decades of intensive research, efficient — or even sub-exponential time — distributionfree PAC learning algorithms are not known for many important Boolean function classes. In this work we suggest a new perspective for algorithmic research on these learning problems, inspired by a surge of recent research in complexity theory, in which the goal is to determine whether and how much of a savings over a naive 2 runtime can be achieved. We establish a range of exploratory results towards this end. In more detail, 1. We first observe that a simple approach building on known uniform-distribution learning results gives non-trivial distribution-free learning algorithms for several well-studied classes including AC, arbitrary functions of a few linear threshold functions (LTFs), and AC augmented with modp gates. 2. Next we present an approach, based on the method of random restrictions from circuit complexity, which can be used to obtain several distribution-free learning algorithms that do not appear to be achievable by approach (1) above. The results achieved in this way include learning algorithms with non-trivial savings for LTF-of-AC circuits and improved savings for learning parity-of-AC circuits. 3. Finally, our third contribution is a generic technique for converting lower bounds proved using Nečiporuk’s method to learning algorithms with non-trivial savings. This technique, which is the most involved of our three approaches, yields distribution-free learning algorithms for a range of classes where previously even non-trivial uniform-distribution learning algorithms were not known; these classes include full-basis formulas, branching programs, span programs, etc. up to some fixed polynomial size. ∗Supported by NSF grants CCF-1420349 and CCF-1563155. †This research was done while visiting Columbia University. Supported by NSF grant CCF-1563122.